Papers with large-scale data curation
From Curated Data to Scalable Models: Continual Pre-training of Dense and MoE Large Language Models for Tibetan (2026.acl-long)
Copied to clipboard
Lei Yang, Leiyu Pan, Bojian Xiong, Renren Jin, Shaowei Zhang, Yue Chen, Ling Shi, Jiang Zhou, Junru Wu, Zhen Wang, Jianxiang Peng, Juesi Xiao, Tianyu Dong, Zhuowen Han, Zhuo Chen, Yuqi Ren, Deyi Xiong
| Challenge: | Large language models have achieved remarkable success across a wide range of tasks, yet their performance remains heavily biased toward high-resource languages. |
| Approach: | They propose a pipeline for advancing Tibetan language modeling through multilingual continual pre-training with Tibetan, Chinese, and English. |
| Outcome: | The proposed model outperforms open-source and Tibetan-focused models on diverse tasks. |